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ABSTRACT 



This document reviews several theoretical frameworks of 
problem-solving, provides a definition of the construct, suggests ways of 
measuring the construct, focuses on issues for assessment, and provides 
specifications for the computer-based assessment of problem solving. As 
defined in the model of the Center for Research on Evaluation, Standards, and 
Student Testing (CRESST) , problem solving is a cognitive process directed at 
achieving a goal when a solution method is not obvious to the problem solver. 
Issues that would drive the assessment specifications for a CRESST assessment 
would be: (1) selection/modification/generation of a conceptual framework; 

(2) what to measure; (3) assessment approaches; (4) criteria; (5) type of 
technology; (6) purpose of testing; (7) type of competency tested; (8) high 
or low stakes level of testing; (9) contexts; and (10) recommended testing 
time. With these points in mind, specifications for the assessment were 
developed, and are presented in table form. The feasibility study previously 
reported (J. Schacter et al . , 1997), which used these specifications, 
illustrated that a Web-based, information-rich environment with these 
specifications can measure student problem solving. An appendix contains the 
scoring key for the developed assessment. (Contains 3 figures and 32 
references.) ( SLD) 
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TEST SPECIFICATIONS FOR PROBLEM SOLVING ASSESSMENT 



Harold F. O'Neil, Jr. 

University of Southern Califomia/CRESST 
John Schacter 
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In this document we review several theoretical frameworks of problem 
solving, provide a definition of the construct, suggest ways of measuring the 
construct, focus discussion on issues for assessment, and provide a specification 
for the computer-based assessment of problem solving. 

In general, there is no widely accepted definition of problem solving that 
has led to a reliable and valid measure of it. The literature on problem solving is 
characterized by multiple theoretical frameworks — cognitive science or socio- 
cultural or information processing. Our theoretical frameworks are twofold: the 
CRESST model of learning and a specific model of problem solving. These 
frameworks are derived from the cognitive science literature. 

The view of problem solving in the cognitive science literature not 
surprisingly when compared with the workplace literature (see O'Neil, 1997) 
tends to be less applied and more theoretical. What is useful, however, in the 
cognitive science literature is a set of distinctions concerning tasks, the role of 
domain knowledge, and a specification of the cognitive strategies /processes used 
in problem solving. These sets of distinctions are listed in Table 1 and are used 
later to focus our assessment of problem solving. 

A synthesis of various theoretical literatures generated the CRESST model 
of learning. The CRESST model includes five families of cognitive learning, of 
which problem solving is one (Baker, 1995). Problem solving is a cognitive 
process directed at achieving a goal when a solution method is not obvious to 
the problem solver (Mayer & Wittrock, 1996, p. 47). 
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Cognitive Science Distinctions in Problem Solving 



Issue 



Definition 



Subprocesses 



Problem-solving 

components 



Problem-solving 

components 



Problem-solving 

process 

Problem-solving 

process 



Math problem- 
solving definitions 



Distinctions 



Problem solving is cognitive processing directed at 
achieving a goal when no solution method is obvious 
to the problem solver, (p. 47) 

Representing, planning, executing 



Identifying the problem, forming an internal 
representation, encoding, planning, strategy 
selection, solution monitoring. 

Domain-specific knowledge (content understanding) 
Metacognition (planning, self-monitoring) 
Domain-specific problem-solving strategies 
Motivation (self-efficacy, effort) 

Cognitive resources, the body of facts and procedures 
capable of being brought to bear in a particular 
mathematical situation. 

Heuristics, ''rules of thumb" of effective problem 
solving, including such aids as drawing figures, 
introducing suitable notation, exploiting related 
problems' analysis, reformulating problems, working 
backwards, and testing and verification procedures. 

Control, having to do with the efficiency, resource 
allocating, and metacognition with which 
individuals utilize the knowledge at their disposal. 

Belief systems, one's perspectives regarding the 
nature of a discipline and how one goes about working 
in it, and level of effort. 

(1) Representing the problem and (2) searching for a 
means to solve the problem. 

(1) Exploration of alternative ideas; (2) extraction of 
relevant material; (3) simplification by constraining 
the problem into parts; (4) organization, attending to 
externally provided feedback. 

A math problem for any student is a task (a) in 
which the student is interested and engaged, and for 
which he wishes to obtain a resolution, and (b) for 
which the student does not have a readily accessible 
mathematical means by which to achieve that 
resolution. 



Reference 



Mayer & Wittrock, 
1996 

Mayer & Wittrock, 
1996 

Sternberg & 
Davidson, 1992 

O'Neil & Schacter 
(this document) 



Schoenfeld, 1989 



Hayes, 1981 
Schacter et al., 1997 

Schoenfeld, 1989 



Table 1 (continued) 



Issue 



Problem solving in 
electronics 



Content-p rocess 
space 

Application areas 

European definitions 
of problem solving 



Mental simulation 



Problem-solving 

definition 



Distinctions 



Lesgold and Lajoie (1989) list the following 
components of problem solving in electronics: an 
analysis of electronics troubleshooting expertise, 
including (a) conceptual underpinnings, principles, 
and models; (b) methods, strategy, and tactics; and 
(c) social, personal and practical knowledge, battle 
experience, war stories, situational tuning. 

Content knowledge (rich, lean) 
vs. process skills (open, constrained) 

Puzzles, math and science, troubleshooting 

There are two principal approaches to studying 
expertise in complex problem solving. One approach 
was largely initiated by Broadbent (1977) in England, 
the other by Dorner (1987) in Germany. The key 
difference between the two traditions is that in the 
former, one is able to specify a precise rule via a 
mathematical formula that would optimize problem 
solving, whereas in the latter, the problems are so 
complex that it is questionable whether we could even 
devise any mathematical or even computer simulation 
that would clearly optimize performance. What the 
two approaches have in common, however, is that one 
can give the problems to anyone of at least roughly 
average intelligence and get them to solve the 
problems, (p. 301) 

A problem-solving strategy is use of mental 
simulation. 

Four primary functions are served by mental 
simulations: generate a course of action, inspect and 
evaluate a course of action, explain a phenomenon, 
and discover and explore models of a phenomenon. 

(p. 337) 

Problem solving is defined broadly here, as higher 
level cognitive activity, either novel or routine, that 
requires previous learning of various types and that 
may result in new learning. . . . Key features of 
problem solving are (1) the task requires a solution or 
sets a performance goal, but the solution process may 
not be a defined procedure, and there may be a 
variety of correct solutions; (2) some degree of search 
takes place in the performer's thinking process; (3) 
the performer uses previously learned rules, verbal 
information, and cognitive strategies to reach a 
solution or achieve the goal; and (4) in the process of 



Reference 



Lesgold & Lajoie, 
1989 



Glaser & Baxter, 
1997 



Sternberg, 1995 



Klein & Crandall, 
1995 



Gagne & Medsker, 
1996 



Table 1 (continued) 



Issue 



Problem-solving 
definition (rich 
content knowledge) 



Domain-independent 

strategies 



General problem- 
solving strategies 



Domain-dependent 

strategies 

(troubleshooting) 

Identifying the 
problem 



Domain-dependent 

strategies 

(troubleshooting) 

Fixing the problem 



Distinctions 



solving the problem, the performer may learn a 
higher order rule or cognitive strategy that will help 
solve similar problems in the future, (pp. 124-125) 

In brief, competent students (a) provide coherent 
explanations based on underlying principles rather 
than descriptions of superficial features or single 
statements of fact; (b) generate a plan for solution 
that is guided by an adequate representation of the 
problem situation and possible procedures and 
outcomes; (c) implement solution strategies that 
reflect relevant goals and subgoals; and (d) monitor 
their actions and flexibly adjust their approach 
based on performance feedback, (pp. 2-3) 

Mental simulations 

Use of analogy 

Use of multiple representations 

Means-ends analysis; Working backward; 
Simplification; Generalization and specialization; 
Trial and error; Rules; Brainstorming; Contradiction; 
Restate the problem; Analogies and metaphors 
(pp. 163-167) 

The sequencing HYDRIVE differentiates between 
several forms of space splitting. There is power 
system elimination, which removes power system 
sources from the problem area (as in checking 
hydraulic pressure gauges or circuit breakers); there 
is active path splitting, which activates different 
combinations of components to achieve a particular 
system function (as in operating the rudders through 
the control stick and through the rudder pedals); and 
there is power path splitting, which either 
eliminates series of edges having the same power 
type or locates the failure to a particular power type 
(as in using electrical backup to replace mechanical 
function). (Adapted from p. 19) 

Removing and replacing a component and observing 
whether the change results in a fix to the system. 

A remove and replace strategy is expensive both in 
terms of time and equipment, and is recommended only 
when there is a high degree of certainty that the 
replaced component is faulty. A serial elimination 
strategy refers to actions that only provide 
information about one component at a time. (Adapted 
from p. 19) 



Reference 



Baxter, Elder, & 
Glaser, 1996 



Klein & Crandall, 
1995; 

Sparks, 1996; 
Anderson, 1990 

Crowl, Kaminsky, & 
Podell, 1997 



Gitomer, Steinberg, & 
Mislevy, 1994 



Gitomer, Steinberg, & 
Mislevy, 1994 



The CRESST model of problem solving (Figure 1) is adapted from the 
problem-solving models of Baxter, Elder, and Glaser (1996), Glaser, Raghavan, 
and Baxter (1992), and Mayer and Wittrock (1996), Sugrue (1995). It includes four 
elements: (a) content understanding, (b) problem-solving strategies (i.e., either 
domain-dependent or domain-independent), (c) metacognition, and (d) 
motivation. An example of a domain-dependent strategy would be a problem- 
solving information-seeking strategy to help a learner find information to better 
solve the problem. For instance, “use Boolean operators" in constructing a search 
argument or browsing using hypertext to survey information spaces. An 
example of a domain-independent strategy would be the use of analogies (e.g., "a 
tire pump is like a syringe" or "electrons around an atom is like the planets 
around the sun"). In general, to be a successful problem solver, one must know 
something (content knowledge), possess intellectual tricks (problem-solving 
strategies), be able to plan and monitor one's progress towards solving the 
problem (metacognition), and be motivated to perform. 

Our R&D in problem solving using the above definition focused on the 
feasibility of using Web-based technology for the assessment of problem solving. 
Currently, the ideal assessment of problem solving is based on think-aloud 
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Figure 1. Model of problem solving. 



protocols (Ericsson & Simon, 1993; Voss & Post, 1988; Voss, Tyler, & Yengo, 1983) 
or performance assessments that require extensive human rater scoring. 
However, such assessments are expensive and time consuming and result in 
delayed (up to months or years) reporting to students, teachers, and parents. In 
our work, we have taken a different approach to measuring these problem- 
solving processes. Instead of conducting interviews, naturalistic observation, or 
think-aloud protocols while students problem solve, we rely on computer (i.e., 
server) access logs to automatically record most of the process behaviors students 
engage in while on the computer. Our approach is to computerize the 
administration, scoring, and reporting of problem-solving measures, thus 
facilitating timely reporting and potentially increasing reliability and validity 
(Herl, O'Neil, Chung, & Dennis, 1997; Martinez, 1993; O'Neil, Chung, & Brown, 
1997; Schacter et al., 1997). 

In our empirical work, content knowledge is assessed by concept maps. 
Domain-specific problem-solving strategies are measured through an 
information-seeking task that analyzes both search behavior and how newly 
found information is utilized. Domain-independent problem solving is assessed 
using metacognition and motivation survey instruments. The basic design 
involves having the student (a) create a concept map, (b) receive feedback on it, 
then, (c) using a simulated Web site, search for information to improve it, and 
(d) construct a final concept map. The final concept map serves as the outcome 
content understanding measure. Finally, metacognition and motivation are 
assessed by paper-and-pencil survey instruments. 

We believe that content understanding and problem-solving strategies are 
best assessed domain-specifically whereas metacognition and motivation are best 
assessed as domain-independent constructs. We have created measures of trait 
metacognition and motivation that can be administered in 4 to 6 minutes and 
would be included in the assessment time for problem solving. The trait scales 
are found in the Appendix. Using constructs from state-trait anxiety theory 
(Spielberger, 1975) as an analogy, we have formulated a set of self-report, 
domain-independent trait and state measures of metacognition and motivation. 
We find the state versus trait distinction useful for both cognitive and affective 
measurement. Thus, we have generalized the key constructs from an affective 
domain (e.g., state and trait anxiety) to a cognitive domain (e.g., state and trait 
metacognition). 







In our technical approach, we have created both process and outcome 
measures of problem-solving strategies. As was mentioned above, we assessed 
problem-solving search behavior by giving students the task of improving their 
existing representational maps (i.e., concept maps). Students were required to 
search for information that would enhance their existing concept maps and, at 
the same time, justify any changes they made to their maps. By monitoring 
students' online search, we can determine whether (a) students search well 
enough to access rich information sources within the "weak" concept areas in 
their maps (as determined from feedback on their initial concept maps); (b) 
students understand enough to define the problem through their search 
behavior; and (c) students can sufficiently justify the changes they make to their 
concept maps. This approach allows us to examine both the searching process 
(through analysis of student search strategies) and the final integrated product 
(the updated concept map). 

Specification Issues in Assessment of Problem Solving 

The purpose of this section is to suggest various approaches for the 
assessment of problem-solving skills. These issues would drive the assessment 
specifications. In general, the issues in assessing these skills are conceptualized as 
(a) selection/modification/generation of a conceptual framework; (b) what to 
measure (e.g., cognitive processes, tasks, or characteristics of jobs and the setting 
or context); (c) assessment approaches (e.g., multiple-choice items, concept map, 
essay, or performances); (d) criteria (e.g., validity, fairness, cost); (e) type of 
technology (e.g., paper-and-pencil, computer); (f) purpose of testing (e.g., 
diagnostic, selection, accountability, credentialling); (g) type of competency tested 
(i.e., individual vs. team); (h) level of stakes (high vs. low); (i) contexts (e.g., 
school, work, home, community); (j) recommended testing time. 

The instantiation of these general assessment issues for CRESST work can 
be seen in Table 2. For example, we use an approach to measure problem solving 
in which domain knowledge, problem-solving strategies, metacognition, and 
motivation are measured as facets of problem solving. Further, the assessment of 
problem solving will be considered to be for accountability purposes of a low- 
stakes nature for the individual involved and will assess only individual 
problem solving. 
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Table 2 



Specification Issues for Assessment of Problem Solving 



Issue 


Problem-solving approach 


Modify a workforce competency 
framework with cognitive science 
definition of problem solving 


SCANS (U.S. Department of Labor, 1992a, 1992b); Mayer 
and Wittrock (1996) 


What to measure 


Domain knowledge, problem-solving strategies, 
metacognition, motivation 


Assessment approach 


Task implemented in a problem-solving block 


Assessment formats 


Concept map, simulated Web space, survey instruments 


Criteria 


Reliability, validity, fairness, and transfer 


Type of task(s) 


Search 


Type of technology 


Computer-based 


Purpose of testing 


Program evaluation 


Type of competency 


Individual and not team 


Level of stakes 


Low for the individual 


Contexts 


Simulate problem-solving tasks at school 


Testing time 


70 minutes 



Assessment Specifications 

The resulting detailed assessment specifications are shown in Figure 2 and 
Figure 3. These are the specifications that we used in our CAETI work. 1 They 
now serve as documentation of the existing assessment software. Figure 2 
contains the specifications for the concept map. Figure 3 contains the 
specifications for the domain-specific, problem-solving search strategy. 



1 The format but not the content of these specifications was adopted from Ruiz-Primo and Shavelson 
(1995). 



General specification attributes 


Specific example 


Task demands 


Construct a concept map with a given set of concepts (terms) and 
links. 


Task constraints 


Student is provided with a fixed set of concepts and links. 

Concepts are important ideas; range from 10-18. The final set of 18 
concepts included atmosphere, bacteria, climate, carbon dioxide, 

DECOMPOSITION, EVAPORATION, FOOD CHAIN, OCEANS, PRODUCERS, 

CONSUMERS, RESPIRATION, SUNLIGHT, PLANTS, PHOTOSYNTHESIS, WASTE, WATER 
CYCLE, OXYGEN, and GREENHOUSE GASES. 

Links are important relationships; range from 6-8. Links used 
were causes, influences, part of, produces, used for, uses. 

Duration: 20 minutes 


Scoring 


Focus is on the following components: 

• propositions and agreement with criterion map(s) 

• criterion maps: compare student's map with experts' map(s) 

Criterion maps were obtained from 4 expert teachers. 


Type of learning 


Content understanding 


Content domain 


Environmental science 


Outcome measures 


Herl metric (Herl, Baker, & Niemi, 1996) comprised of: 

Semantic content score, organizational structure score, 
number of terms used, number of links used, comparison to 4 
expert maps 


Prerequisite skills 


None 


Training 


8 minutes of instruction 


Cognitive processes 
(domain-dependent) 


Representation of content knowledge 


Cognitive processes 
(domain-independent) 


Metacognition 


Affective processes 
(domain-independent) 


Effort and self-efficacy 


Student response 


The student response will be on a computer. 


Hardware and software 
components 


Macintosh Power PC or Pentium PC; Mac OS 7.5.3 or Windows 95 
or NT; 16 megabytes of RAM or more; 20 megabytes of hard disk 
space; Netscape 3.0 (Java and Javascript capable browser); Open 
Transport 1.1 or TCP/IP; Ethernet; programming language (Java, 
HyperCard); 


Language 


English 



Figure 2. Domain specifications for individually constructed concept map. 



General specification attributes 



Specific example 



Task demands 


Use Netscape and a simulated Web space to find relevant 
information via searching to improve existing concept map (see 
Figure 2). Bookmark relevant information to justify new concepts 
and links. 


Task constraints 


Student are provided with a simulated Web environment: 

• Our Web server software is AOL. Environment consists of 130 
Web documents and over 400 images and diagrams about 
environmental science and other topic areas. 

• The database is made up of both relevant and irrelevant 
information about environmental science that varies in degree 
of specificity, reading level, scope and comprehensiveness. 
Ninety percent of the information on the simulated Web was 
downloaded from the WWW using Web Whacker 1.0 software. 
The other 10% of the information was adapted from science 
textbooks and magazines. 

• Four mechanisms to search a simulated Internet information 
space: (a) search engine (b) hierarchical subject listing 
(directory), (c) hypertext, and (d) a glossary of environmental 
science terms. 

• A mechanism to bookmark Web pages encountered during 
information seeking and send those bookmarks to nodes in the 
map. 

Duration: 35 minutes 


Scoring 




Scoring problem-solving 
outcomes 


Outcome of map score after searching; premap to postmap 
improvement 


Relevance-rated database 


Two experts rate all information in database on 4-point scale for 
its relevance to each concept in the task of constructing a concept 
map. 


Extraction 


Student bookmarks for each concept are compared to expert 
relevance ratings for each concept with scope and depth scores. 


Simplification 


Students searched on at least 20% of the concepts that they 
needed to work on. 


Exploration 


• Frequency counts of Boolean searching, synonym substitution, 
multiple terms in searching string 

• Frequency counts of total search moves, browsing moves, scan and 
select moves, analytic queries made 


Organization 


Use of map feedback 



Figure 3. Specifications for domain-specific problem-solving search strategy. 



Type of learning 


Problem-solving strategies 


Content domain 


Environmental science 


Outcome measures 


(a) Herl metric (Herl, Baker, & Niemi, 1996); (b) student 
bookmarks 


Prerequisite skills 


Computer concept mapping 


Training 


12 minutes of instruction 


Cognitive processes 
(domain-dependent) 


Problem-solving processes (exploration, extraction, 
simplification, and organization) 


Cognitive processes 
(domain-independent) 


Metacognition 


Motivational processes 
(domain-independent) 


Effort, self-efficacy 


Student response 


The student response will be on a computer. 


Hardware and software 
components 


Macintosh Power PC or Pentium PC; Mac OS 7.5.3 or Windows 95 
or NT; 16 megabytes of RAM or more; 20 megabytes of hard disk 
space; Netscape 3.0 (Java and Javascript capable browser; open 
transport 1.1 or TCP/IP; Ethernet; programming language (Java, 
HyperCard) 


Language 


English 



Figure 3 (continued). 



Feasibility of a Web-Based Assessment of Problem Solving 

In the feasibility study reported at the 1997 annual meeting of the American 
Educational Research Association (Schacter et al., 1997), which used the above 
specifications, we presented our first attempt at automated data collection and 
scoring of students' complex problem-solving processes and performance in 
Web-based, information-rich environments We are still developing the real- 
time scoring and reporting software. In general we found the computer-based 
problem-solving assessment to be feasible. 

The feasibility study illustrated that a Web-based, information-rich 
environment can measure student problem solving. We argue that this finding 
is promising because it provides evidence that we can measure problem solving 
when working on realistic problems in realistic contexts that demand the 
activation of multiple cognitive processes. Such measurement is facilitated by 
the use of specifications. 
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APPENDIX 



CAETI SCORING KEY (25 August 1996) 



TRAIT THINKING QUESTIONNAIRE 



Scales 



Items 



Planning 

Cognitive strategy 
Self-checking 



1, 8, 11, 19, 25, 28, 33, 36 



2, 7, 12, 17, 21, 29, 34, 38 



3, 6, 13, 16, 22, 30, 35, 39 



Effort 



4, 9, 14, 18, 23, 26, 31, 40 



Self-efficacy 



5, 10, 15, 20, 24, 27, 32, 37 



PLANNING 

1 . I determine how to solve a task before I begin. 

8. I carefully plan my course of action. 

11. I try to understand tasks before I attempt to solve them. 

19. I try to understand the goal of a task before I attempt to answer. 

25. I figure out my goals and what I need to do to accomplish them. 

28. I imagine the parts of a task I have to complete. 

33. I make sure I understand just what has to be done and how to do it. 
36. I try to determine what the task requires. 

COGNITIVE STRATEGY 

2. To understand a task, I draw a graph if at all possible. 

7. I think through the steps of a plan in my mind. 

12. While solving a task, I try more than one way to do it. 

17. I think through the meaning of tasks before I begin to answer them. 
21. I select and organize relevant information to solve a task. 

29. I spend more time trying to understand difficult tasks. 

34. I attempt to discover the main ideas in a task. 

38. I ask myself how this task relates to what I already know. 
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SELF-CHECKING 

3. I check how well I am doing when I solve a task. 

6. I ask myself questions to stay on track as I do a task. 

13. I check my work while lam doing it. 

16. I almost always know how much of a task I have to complete. 

22. I judge the correctness of my work. 

30. I correct my errors. 

35 . I check my accuracy as I progress through a task. 

39. I ask myself, how well am I doing, as I proceed through tasks. 

EFFORT 

4. I work hard to do well even if I don't like a task. 

9. I put forth my best effort on tasks. 

14. I work as hard as possible on tasks. 

18. I am willing to do extra work on tasks to improve my knowledge. 

23. I concentrate as hard as I can when doing a task. 

26. I work hard on a task even if it does not count. 

31 . A task is useful to check my knowledge. 

40. Practice makes perfect. 

SELF-EFFICACY 

5 . I believe I will receive an excellent grade in this course. 

10. I'm certain I can understand the most difficult material presented in the readings 
for this course. 

15. I'm confident I can understand the basic concepts taught in this course. 

20. I'm confident I can understand the most complex material presented by the 
teacher in this course. 

24. I'm confident I can do an excellent job on the assignments and tests in this course. 

27. I expect to do well in this course. 

32 . I'm certain I can master the skills being taught in this course. 

37. Considering the difficulty of this course, the teacher, and my skills, I think I will 
do well in this course. 
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Trait Thinking Questionnaire 



Name (please print): 
Teacher: 



Directions : A number of statements which people have used to describe themselves are given below. 
Read each statement and indicate how you generally think or feel on learning tasks by marking your 
answer sheet. There are no right or wrong answers. Do not spend too much time on any one 
statement. Remember, give the answer that seems to describe how you generally think or feel. 



Almost Almost 

never Sometimes Often always 



1 . I determine how to solve a task before I begin. 1 

2. To understand a task, I draw a graph if at all 1 

possible. 

3 . I check how well I am doing when I solve a task. 1 

4. I work hard to do well even if I don't like a task. 1 

5 . I believe I will receive an excellent grade in this 1 

course. 

6. I ask myself questions to stay on track as I do a 1 

task. 

7. I think through the steps of a plan in my mind. 1 

8. I carefully plan my course of action. 1 

9. I put forth my best effort on tasks. 1 

10. I'm certain I can understand the most difficult 1 

material presented in the readings for this course. 

11. I try to understand tasks before I attempt to solve 1 

them. 

12. While solving a task, I try more than one way to do 1 

it. 

13. I check my work while I am doing it. 1 

14. I work as hard as possible on tasks 1 

15. I'm confident I can understand the basic concepts 1 

taught in this course. 

16. I almost always know how much of a task I have to 1 

complete. 

17. I think through the meaning of tasks before I begin to 1 
answer them. 



2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 



2 3 4 
2 3 4 
2 3 4 



2 3 4 



2 3 4 
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17 



21 



18. 

19. 

20. 
21. 

22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 



Almost 

never Sometimes Often 



I am willing to do extra work on tasks to improve 1 

my knowledge 

I try to understand the goal of a task before I 1 

attempt to answer. 

I'm confident I can understand the most complex 1 

material presented by the teacher in this course. 

I select and organize relevant information to solve a 1 
task. 

I judge the correctness of my work. 1 

I concentrate as hard as I can when doing a task. 1 

I'm confident I can do an excellent job on the 1 

assignments and tests in this course. 

I figure out my goals and what I need to do to 1 

accomplish them. 

I work hard on a task even if it does not count. 1 

I expect to do well in this course. 1 

I imagine the parts of a task I have to complete. 1 

I spend more time trying to understand difficult 1 

tasks. 

I correct my errors. 1 

A task is useful to check my knowledge. 1 

I'm certain I can master the skills being taught in this 1 
course. 

I make sure I understand just what has to be done 1 

and how to do it. 

I attempt to discover the main ideas in a task. 1 

I check my accuracy as I progress through a task. 1 

I try to determine what the task requires. 1 

Considering the difficulty of this course, the teacher, 1 
and my skills, I think I will do well in this course. 

I ask myself how this task relates to what I already 1 
know. 

I ask myself, how well am I doing, as I proceed 1 

through tasks. 

Practice makes perfect. 1 



2 3 

2 3 

2 3 

2 3 



2 3 

2 3 

2 3 



2 3 



2 3 

2 3 

2 3 

2 3 



2 3 

2 3 

2 3 



2 3 



2 3 

2 3 

2 3 

2 3 



2 3 



2 3 

2 3 




Copyright © 1996 Harold F. O'Neil, Jr. 




Almost 

always 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 
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